POLAR AND COISOTROPIC 
ACTIONS ON KAHLER MANIFOLDS 



Fabio Podesta and Gudlaugur Thorbergsson 

Abstract. The main result of the paper is that a polar action on a compact irre- 
ducible homogeneous Kahler manifold is coisotropic. This is then used to give new 
examples of polar actions and to classify coisotropic and polar actions on quadrics. 



§1. Introduction 

The aim of the present paper is to investigate the relationship between polar and 
coisotropic actions on compact Kahler manifolds. 

The action of a compact Lie group G of isometrics on a Riemannian manifold 
(M, g) is called polar if there exists a properly embedded submanifold S which 
meets every G-orbit and is orthogonal to the G-orbits in all common points. Such 
a submanifold S is called a section (see [PT1,2]) and if it is flat, the action is called 
hyperpolar. It is of course meaningful to relax the definition of polar actions and not 
require that the section is properly embedded. Our results are correct for weakly 
polar actions in this sense, making Theorems 1.1 and 1.3 stronger, but Theorem 
1.2 weaker. 

If (M, g) is a compact Kahler manifold with Kahler form uj and G is a compact 
subgroup of its full isometry group, then the G- action is called multiplicity-free 
([GS2]) or coisotropic ([HW]) if the principal G-orbits are coisotropic with respect 
to Lo. Notice that the existence of one coisotropic principal G-orbit implies the same 
property for all principal G-orbits, see [HW]. 

In this paper we shall consider the special case of M being a simply connected, 
compact homogeneous Kahler manifold; such manifolds are diffeomorphic to quo- 
tient spaces L/K, where L is a compact semisimple Lie group and K is the cen- 
tralizer in L of some torus. We also recall that a compact homogeneous Kahler 
manifold cannot be written as a nontrivial Riemannian product of two Riemannian 
manifolds if and only if its full isometry group is a compact simple Lie group; in 
this last case the manifold is simply connected (see e.g. [On], p. 238 ff). 

In section two we will prove the following 
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Theorem 1.1. A polar action on an irreducible compact homogeneous Kdhler man- 
ifold is coisotropic. 

It is obvious that the theorem is not true without the assumption that M is 
irreducible. The converse of Theorem 1.1 is not true. In fact, we will give an 
example of a coisotropic action on Pn(C) which is not polar in section three and 
there are further such examples on quadrics in Theorem 1.3. In certain situations, 
we can prove that coisotropic actions are polar and we will use this to give many 
new examples of polar actions. We will for example show that every nonoriented 
compact surface N is the section of a polar action on some four dimensional compact 
Kahlcr manifold. In section three we will prove the following 

Theorem 1.2. An effective and isometric action of the (real) torus on a com- 
pact Kdhler manifold M with positive Euler characteristic and complex dimension 
n is polar. 

The condition on the Euler characteristic is to ensure that the action is Poisson; 
see the proof in section three. We also remark that this condition is not restrictive, 
since a compact, symplectic 2n-dimensional manifold which is acted on effectively 
by an n-dimensional torus in a Poisson fashion has positive Euler characteristic (see 



As an application of Theorem 1.1, we will classify polar actions on quadrics 
in section four. We do this by first classifying the coisotropic actions and then 
deciding which of them are polar. An interesting consequence of this classification 
is that polar actions on quadrics are hyperpolar. This is in contrast with complex 
projective spaces or more generally to rank one symmetric spaces that admit many 
polar actions that are not hyperpolar (see [PTh]). Another interesting consequence 
of the classification is that all coisotropic actions on quadrics are related to transitive 
actions on spheres. The classification of coisotropic and polar actions on quadrics 
is given in the following 

Theorem 1.3. Let G he a connected compact Lie subgroup o/SO(n) (n > 5) and 
let Qn-2 be the quadric S0(n)/S0(2) x SO(n — 2). Then the action of G on Qn-2 
is nontransitive and coisotropic if and only if one of the following occurs. 

(i) The action of G on is irreducible and G is conjugate to 



Only the action of the last one of these groups is not polar on Qn-2- 
(ii) The action of G on has a one dimensional fixed point set and the group 
G as a subgroup ofSO{n — 1) is conjugate to one of the following 
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SU(^), U(^), Sp(l)-Sp(^), Ti.Sp(n/4), Spin(9). 



SO(n-l), G2, Spin(7), 
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Only the actions of the first three of these groups are polar on Qn-2 ■ 
(iii) The action of G on is reducible and M" is the sum of two irreducible 
nontrivial G-submodules Vi,V2; the group G splits as the product G\ x G2 
with Gi C SO (14), i = 1,2 and each Gi is conjugate in SO{Vi) to one of the 
following 

SO{p), G2, Spin(7), Sp(l)-Sp(f>), Spin(9) 

for suitable p G N. Only the actions obtained choosing each G^, i = 1,2, 

among the first three of these groups are polar on Qn-2- 

Moreover if the G-action on Qn-2 is polar, then it is hyperpolar. 

We now recall some basic definition and results from the paper [HW]. A sub- 
manifold of a symplectic manifold (M, oj) is called coisotropic if 

TpW^" C TpW 

for all p in W . Here TpW-^'^ denotes the subspace of TpM that is w-orthogonal to 
TpW. If M is a Kahler manifold and its Kahler form, then it is easy to see that 
a submanifold W of M is coisotropic if and only if 

J{NpW) C TpW 

for all p in W, where J denotes the complex structure of M, and NpW the normal 
space of W in p. One calls a real subspace V in a, complex vector space with a 
Hermitian scalar product totally real if V is perpendicular to J{V). It follows from 
this discussion that a submanifold W in a Kahler manifold is coisotropic if and only 
if all its normal spaces are totally real. 

Let (M, uj) be a symplectic manifold. Then one can associate to every function 
/ in C°°(M) its symplectic gradient Xj defined as the unique vector field on M 
such that 

df{Y)=io{Xf,Y) 

for every vector field Y on M. If / and g are two functions in C°°, then one 
defines their Poisson bracket as {f,g} = u{Xf,Xg). It follows that C°°{M) with 
the Poisson bracket is a Lie algebra. 

Now let G act on M preserving the symplectic structure u. Then the action is 
called Poisson if there is a Lie algebra homomorphism A : g ^ C°°{M) such that 
Xx(^) agrees with the infinitesimal action of on M. The moment map of a Poisson 
action is defined as 

$:M^0*; $(p)(e) = A(0(p). 

We have ^{gp) = Ad*{g)^{p), i.e., the moment map is G-equivariant. 

Let G be a compact group acting isometrically on a compact Kahler manifold. 
This action is automatically holomorphic by a theorem of Kostant (see [KN], vol. I, 
p. 247) and it induces by compactness of M an action of the complexified group 
on M. We say that M is G'^-almost homogeneous if G'^ has an open orbit in M. If 
all Borel subgroups of act with an open orbit on M, then the open orbit f2 in 
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M under the action of G is called a spherical homogeneous space and M is called 
a spherical embedding of fJ. In general, given a complex Lie group U and a closed 
subgroup H (Z U, the pair ([/, H) is called a spherical pair if every Borel subgroup 
of U has an open orbit in U/H; all such pairs {U,H), where U is semisimple and 
H is reductive, have been classified by Brion ([Br2]). 

One of the basic results on spherical embeddings is that if the orbit Q of is a 
spherical homogeneous space, then G'^ and all Borel subgroups of G'^ have finitely 
many orbits in M, see [Brl]. 

The following theorem is proved in [HW], p. 275. 

Theorem 1.4 (Equivalence Theorem of [HW]). Let M be a connected compact 

Kdhler m^anifold with an isometric action of a connected compact group G that is 
also Poisson. Then the following conditions are equivalent: 

(i) The space C°°{M)^ of G -invariant functions on M is abelian with respect 
to the Poisson bracket. 

(ii) The G-action is coisotropic. 

(iii) The cohomogeneity of the G-action is equal to the difference between the 
rank of G and the rank of a regular isotropy subgroup of G. 

(iv) The moment map $ : M — >■ separates orbits. 

(v) The Kdhler manifold M is projective algebraic, G^-almost homogeneous and 
a spherical embedding of the open G"^ -orbit. 

We remark here that the conditions (i) to (iii) are equivalent even without the 
hypothesis of compactness for M (see [HW] ) . 

Under the hypothesis of Theorem 1.4, the following convexity theorem holds (see 
[GSl] and [Kir]): if T denotes a maximal torus of G, t* is the fixed point set of T 
in 0* and i\ denotes a fundamental domain for the action of the Weyl group on 
t*, then the image $(M) under the moment mapping $ intersects t!j_ in a convex 
polytope. 

On the other hand, if G is a compact group of isometrics of a Riemannian 
manifold and if the G-action is polar, then we can associate to any section S a 
finite group W, also called Weyl group, which is defined as W := Ng{T,) /Zg{^)', 
here NgC^) (resp. Zg(I])) denotes the subgroup of G given by those elements which 
map S onto itself (resp. act trivially on S). It is well known that the intersection of 
a G-orbit with S coincides with a H-^-orbit and we can identify the two orbit spaces 
M/G and S/VF; for general properties of the Weyl group we refer to [PTl], [PTh], 
[Co]. 

^Prom Theorem 1.1 and Theorem 1.4 we obtain the following 

Corollary 1.5. Let M be an irreducible homogeneous Kdhler manifold which is 
acted on polarly by a compact Lie group of isometrics G. Then 

(1) the cohomogeneity of the G-action does not exceed the rank ofG; moreover 
the complexified group G^ and any Borel subgroup of G'^ act on M with 
finitely many orbits; 

(2) if ^ : M Q* denotes the moment map, S is any section for the G-action 
and W its Weyl group, then the moment map induces a homeomorphism 
between S/W and the convex polytope $(M) n t^. 
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To prove the corollary one only has to observe that the action of G is Poisson 
since the full group of isometrics of M acts in a Poisson fashion: indeed any flag 
manifold is a coadjoint orbit whose moment map is simply the inclusion. 

We end this section pointing out one common property of polar and coisotropic 
linear actions of a compact Lie group G which will be used in section four. It is 
known that a polar linear action of G on a (real) vector space does not contain two 
distinct irreducible equivalent G-submodules (see [HPTT], Proposition 2.10); if G 
is realized as a subgroup of U(A^) for some N > 1 and if the corresponding action 
on C''^ is coisotropic with respect to the standard symplectic form of C^, then 
every irreducible complex G-module appears in with multiplicity at most one, 
i.e. every linear coisotropic action of G is multiplicity free (see for instance [HW], 
Theorem 1, p. 275). We also note here that the restriction of a linear coisotropic 
action to a complex G-submodulc is still coisotropic (see the Equivariant Mapping 
Lemma in [HW], p. 272); as a consequence, a linear coisotropic action has no 
nontrivial fixed vectors, in contrast with the polar actions. 

The following result states that a coisotropic linear action is multiplicity free 
also from the 'real' point of view. 

Proposition 1.6. Let G be a compact subgroup ofJJ{N) for some N > I. If 
the corresponding action of G on is coisotropic with respect to the standard 
symplectic form on , then the real G-module ~ does not contain two 
distinct equivalent G-submodules. 

The proof of this proposition will be given in the next section. 
§2. Proofs of Theorem 1.1 and Proposition 1.6 

Let M be a compact, irreducible homogeneous Kahler manifold with Kahler 
metric g and complex structure J; if we denote by L the identity component of the 
full isometry group of {M,g), then L preserves the complex structure J (see the 
introduction) and we can represent M as M = L/K, where K is the centralizer in 
L of a torus. Note again that the irreducibility of (M, g) implies that L is simple. 

We consider a compact connected Lie subgroup G C L of holomorphic isometrics 
and we shall suppose that the G-action on M is polar with a section S. 

We saw in the introduction that a submanifold of a Kahler manifold is coisotropic 
if and only if all of its normal spaces are totally real. The proof of Theorem 1.1 is 
therefore a direct consequence of the following proposition. 

Proposition 2.1. The section S is totally real. 

The proof of this proposition requires some lemmas. 

We assume that the cohomogeneity of the action is at least two since the claim 
in the proposition is trivially true for cohomogeneity one actions. First of all, if S 
is a section, we may define the complex subspace 

Hp = TpT, n JTpS (2.1) 

for p G S. We next prove that H is a, complex subbundle over S. It suffices to 
show that if p and q are two points in S and a a path in S joining them, then 
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parallel transport along a sends Hp to Hq . This is clear since E is totally geodesic 
and parallel transport in M commutes with J. An important consequence of this 
argument that we will use later is that 7i \s a parallel subbundle of T(I]). 

We can now define a complex subbundle H of T{M)\m,^^ by setting Tiq = 
TqT,{q) n JTqT,{q) for q G M^eg, where denotes the unique section through 
q and M^^g denotes the set of regular points of the G-action. 

Our aim is to prove that Tip = {0} for all p. In order to do this, we will first 
show that H extends to a smooth integrable subbundle of the whole T{M), whose 
orthogonal complement is also integrable, and then we will use a result on foliations 
to prove our claim. 

Lemma 2.2. The subbundle H can he extended to a differentiable subbundle of the 
tangent bundle T{M). 

Proof. We consider a nonregular point q ^ M and fix two sections passing 
through q. If we denote by 'Hg(S) and Hq{T,') the complex subspaces of TqT, and 
TgS' respectively, defined as in (2.1), we first must prove that they coincide and 
then that the extension of the bundle to the whole manifold M is differentiable. 

We will use the fact that the isotropy representation of Gq is polar (see [PT1,2]). 
We start proving the following 

Sublemma 2.3. The subspace 7iq{T,) is contained in the intersection of all singular 
hyperplanes of TqT, with respect to the isotropy representation of Gp. 

Proof. Indeed, if F C TqT, is a singular hyperplane, then there exists an element 
g & Gq which leaves S invariant and such that dgq is a reflection in Y. Since 
g is holomorphic, it leaves the maximal holomorphic subspace of TqT invariant; 
moreover, since the fixed point set Y has real codimension one, it must contain 

HqiT). □ 

Sublemma 2.4. Let G be a compact Lie group acting linearly and, polarly on a 
vector space V . IfT and T' are two different sections with nontrivial intersection, 
then the intersection S n E' contains the intersection of all singular hyperplanes of 
T (and ofT'). 

Proof. We use induction on the dimension of V, the case when diml/ = 2 being 
trivially true. First of all, we may suppose that the fixed point set V*^ is equal to 
{0}. Indeed, if {0}, we decompose V = (S)W, with W = (V^)-^ and we 

note that any section splits as the sum of V'^ plus a section for the G-action in 
W; using the induction hypothesis on W, we get our claim. So, we suppose that 
= {0}. If we fix a nonzero vector v eTHT', we clearly have that v is singular 
for the G-action and moreover the orbit Gv has positive dimension. Therefore 
the isotropy G„ acts polarly on the normal space N,^ to the orbit Gv and, since 
dim Ny < dim V, by the induction hypothesis, we have that T ClT' contains the 
intersection of all singular hyperplanes in T passing through v. A fortiori, it then 
follows that TnT' contains the intersection of all singular hyperplanes in E. □ 

We now extend the bundle H to the singular points. We fix a singular point q 
and consider two sections E, E' through q. They correspond to two linear sections. 
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also denoted by S, S', in the normal spaces Nq of the orbit Gq for the polar action 
of the isotropy subgroup Gq. The JC-cycles of Bott and Samelson ([BS]) can be 
used to prove that there is a sequence of linear sections Sq = S, . . . , = S' such 
that Sj n is nontrivial since we are assuming cohomogeneity greater than 

one (see the proof of Lemma IB. 3 in [PTh]). Prom Sublemma 2.4, we see that 
Si n contains the intersection of all singular hyperplanes of and therefore, 
by Sublemma 2.3, it contains Hq{T,i); similarly, it contains 7^q(Si_|_i). Since 7iq(Ej) 
is the maximal complex subspace of Sj, it follows that HqC^i+i) C Hq{T.i) and we 
also have the opposite inclusion, so that Hq{T,i) = HqC^i+i). Therefore = 

We are left with proving that the extension is differentiable. 

We fix a singular point q E M and a basis {vi, ...,«;} of Hq-, we will show that 
there exist smooth vector fields v^,. . . ,v* on some suitable neighborhood U of g, 
which extend vi, . . . ,vi respectively and span Tly for all y & U. 

We denote by Nq the normal space Tq{G ■ q)^ and choose a real positive number 
r > so that exp^ : Br ^ M is a diffeomorphism onto its image, where 
Br ■■= {v G Nq] \\v\\ < r}. We set Afr := expq{Br) and note that Afr = [jAfr n E, 
where the union is taken over all sections S passing through q. If y G Mr, then 
y = exp^(y) for some Y e Br and for i = 1, . . . , / we define Vi{y) to be the parallel 
transport of Vi along the geodesic expq{tY), t G [0, 1]; note that if S is any section 
through q with Y G T^S, then y G E and Vi{y) e Hy C TyT, C TyMr, since H is 
parallel along the sections. 

Let now Qq be the Lie algebra of the isotropy subgroup Gq and fix a complement 
rriq of Qq in g so that g = + m,; it is clear that, if r is sufficiently small, there 
exists a neighborhood F of O G nVq so that the map 

V xMr^M 

(X,y)^exp^(X)-y 

is a diff'eomorphism onto an open neighborhood U oi q \u M. If x G C/, say 
X = exp'^(X) • y with X eV and y eAfr,we define v*{x) = exp*^(X)*(-Oj(y)) G Hx 
for alH = 1, . . . , Z and this proves our claim. 

We now prove the following 

Sublemma 2.5. The distribution given by 7i is integrable and totally geodesic; the 
distribution given by is integrable. 

Proof. It is enough to prove our claims on the regular set M^eg which is dense in 
M. 

The claims about H follow from the fact that 7^ is a parallel subbundle of T(E) 
for all sections E. 

We now put k = dim E — dimH and m the dimension of a regular G-orbit. Then 
dimTY^ = k+m. We pick a regular point q and consider the section E through g; we 
then fix a set {Zi, . . . , Z^} of smooth independent sections of which are tangent 
to E over a suitable neighborhood F n E, where V is an open neighborhood of q in 
Mreg. We may extend the sections {Zi,. . . ,Zk}to vector fields on a neighborhood of 
q by using the G-action, since the slice representation of Gp is trivial for every p eV. 
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On some suitable open neighborhood V of q in V, we can choose Killing vector 
fields {Xi, . . . , Xjn}, induced by the G-action, which span the tangent space to every 
orbit Gp fovp G V; the vector fields {Zi, . . . , Zk,Xi, . . . ,Xm} are therefore linearly 
independent and span Hp for all p G V. We now prove that H-^ is integrable. First 
of all we note that, by the definition of Zi the Lie derivative Cxj Zi =0 and therefore 
[Zi,Xj] = for all i = 1, . . . , fc, j = 1, . . . ,m; moreover, [Xi, Xj] is tangent to the 
G-orbits and is therefore a section of 7^-'- for all i, j = 1, . . . ,m. It is left to prove 
that [Zi, Zj] are sections of for all i,j = 1, ... ,k. 
Given any section X G T{'H) on V, we have 

g{[Zi,Zj],X) = g{Zi,Vz,X) - g{Zj,Vz,X), 

where V denotes the Levi-Civita connection of the metric g. If x G V and T,{x) 
denotes the (unique) section through x, then Zi,Zj and X are tangent to T,{x) 
and V ZiXjV z -X are sections of 7i, since H is a parallel subbundle of T(T,{x)); 
therefore g{[Zi, Zj], X) = and our claim follows. □ 

Lemma 2.6. We have Hp = {0} for all p e M. 

Proof. In [BH] the following result is proved: given a totally geodesic foliation 

on a compact Riemannian manifold (M, g) such that the distribution given by 
TJ^-^ is integrable, then the universal covering manifold M splits topologically 
as the product of two manifolds that are the universal coverings of leaves of the 
two distributions and JT-"- . In our situation, the manifold M is simply connected; 
moreover, it is known (sec [On] , p. 279 and [Sh]) that the cohomology ring H* (M, M) 
is indecomposable, i.e. can not be written as a tensor product A(Si B, where A, B 
are differential graded algebras of dimension bigger than two. This implies that the 
leaves of the distribution H are zero dimensional. □ 

Proof of Proposition 2.1. Since the manifold M is compact, the complexified Lie 
group acts on M. Moreover, we can consider an Iwasawa decomposition = 
g +s, where s is a solvable Lie subalgcbra. We recall that the manifold M = L/K 
can be L'^-equivariantly embedded into some complex projective space. If S denotes 
the Lie subgroup of with Lie algebra s, then by the Borel Fixed Point Theorem, 
S has a fixed point in M and therefore there exists a point q G M such that the 
G^'-orbit G'^q coincides with the G-orbit Gq, which is therefore complex. We denote 
by Nq the normal space of the orbit Gq at q; then Ng is a complex subspace of 
TqM and the isotropy Gq acts on it polar ly with a section S. 

First of all, we note that the fixed point set Nq " is equal to {0}, since otherwise 
it would be a complex subspace contained in a section, contradicting Lemma 2.6. 
Therefore the Gg-module Nq splits as a sum of irreducible, nontrivial complex 
submodules; these are also irreducible over the reals since otherwise Nq would 
contain two equivalent real submodules, which is not possible by [Da], see also 
[HPTT], p. 168. Our claim will now follow from the following lemma. 

Lemma 2.7. Let K he a compact Lie subgroup of 0{V), where V is a real vector 
space. If the action of K onV is irreducible, polar and leaves some complex struc- 
ture J on V invariant, then the sections are totally real subspaces with respect to 
J. 
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Proof. We know from Dadok's Theorem that the i^-action is orbit equivalent to the 

isotropy representation of a symmetric space U/H and that K can be considered 
to be a subgroup of H. Furthermore the representation of -fC on y is the restriction 
of the isotropy representation oi U/H. UK itself is the isotropy subgroup of a 
symmetric space U/K, then U/K is Hermitian symmetric and the sections are 
totally real: indeed if u = ^ + m is the corresponding Cartan decomposition, we 
identify V with m and the complex structure J can be represented as the adjoint 
action of some element z ^ therefore, if S is any section and v^w G S, then 
< Jv^w >=< [z, >= 0, since w belongs to S which is orthogonal to the 
tangent space of the orbit Ad(i^)-orbit through v. 

If, on the other hand, there exists a symmetric pair ([/, H) such that if is a 
proper subgroup of H and H, K have the same orbits in V, where u = ^ + V is the 
corresponding Cartan decomposition, then the pairs {K, H) have been completely 
classified in [EH] (see also [GT]). We note here that our claim is obviously true 
when the cohomogeneity is one. The list of such pairs (if, H) with cohomogeneity 
of the if -action at least two is given in the following table, where we also indicate 
the group such that (C/, H) is a symmetric pair. 



K 


H 


u 


SO(2) X G2 


SO(2) X SO(7) 


SO(9) 


SO(2) X Spin(7) 


SO(2) X SO(8) 


SO(IO) 


SO(3) X Spin(7) 


SO(3) X SO(8) 


SO(ll) 


SU(p) xSU(g), p/g 


5(U(p) X U(?)) 


SU(p + q) 


SU(n), n odd 


U(n) 


SO(2n) 


SO(IO) 


• SO(IO) 


Eg 



Since the corresponding symmetric spaces U/H are Hermitian symmetric with 
the only exception of if = S0(3) x S0(8) and K = S0(3) x Spin(7), it will be 
enough to prove that the representation space V = R^^ of the group K (here the 
representation is the tensor product of the standard representation of SO (3) on 
and the spin representation of Spin(7) on R^) does not carry any invariant complex 
structure. If V carries such a complex structure, then K would have an irreducible 
representation in C^^, which would be a tensor product of two irreducible represen- 
tations of S0(3) and Spin(7) respectively; this contradicts the fact that the least 
dimensional irreducible complex representation of Spin(7) is seven dimensional. □ 

Remark. We point out here that the homogeneity of the Kahler manifold M was 
used only in the proof of Lemma 2.6 and at the beginning of the proof of Proposition 
2.1 where we use the Borel Fixed Point Theorem. Actually, looking at the proof 
of Lemma 2.6, we see that Theorem 1.1 holds true for every compact, projective 
algebraic Kahler manifold whose universal covering space is not diffeomorphic to a 
product of two Kahler manifolds of lower dimension. 

Proof of Proposition 1.6. We denote by J the complex structure and by <, > the 
scalar product on the real G-module R^-^. We split = Yl!i=i ^ the sum 
of mutually inequivalent complex irreducible G-submodules; we recall that each 
{Wi)M., i.e. the real G-module obtained from Wi by restricting the scalars, is either 



10 



irreducible or splits as the sum of two equivalent submodules which are interchanged 
by J. Suppose that the G-modulc contains two equivalent irreducible G- 

submodules; then we can suppose that there exist two equivalent G-submodules 
El , E2 such that either Ei , E2 are both complex or Ei + E2 is complex and J 
interchanges Ei,E2. Since the restriction of the G-action to a complex submodule 
is still coisotropic we can restrict ourselves to the complex submodule W := Ei + 
E2. We identify both Ei,E2 with the irreducible G-module V endowed with the 
invariant scalar product g and W with the orthogonal sum V © V so that < , > 
restricts to g on the first factor and with c^g on the second one for c G R \ {0}. 
If El and E2 are not complex submodules, then Hom{V,V)^ = M (see e.g. [BtD], 
p. 99) and it is easy to see that J is given up to the sign by J{v, w) = {—cw, ^v) for 
v,w eV; if both Ei,E2 are complex submodules, then V has a complex structure 
J and J{v,w) = {Jv, —Jw), since Ei ~ E2 as complex modules. 

We now construct two functions /i, /2 G C°°{W)^ which do not commute with 
respect to the Poisson bracket, contradicting Theorem 1.4. We define fi{v,w) = 
g{v,v) and f2{v,w) = g{v,w) for v,w eV. Then XjJ(^^^) is given by {2Jv,0) if 
V is complex and by (0, if JEi = E2. It is then easy to check that {/i, /2} = 
—df2{Xf^) does not vanish identically, proving our claim. □ 

§3. New examples of polar actions 

In this section we first prove Theorem 1.2 and then give new examples of polar 
actions. We also show that there is a coisotropic action which is not polar. 

Proof of Theorem 1.2. We will denote by J and lo the complex structure and the 
Kahler form of M respectively and denote the torus by G. The commutativity of 
G implies that the action has only one principal isotropy group which is contained 
in all other isotropy groups. It follows that the principal isotropy group is trivial 
since the action is effective. Hence the principal orbits are n-dimensional. Since 
the Euler characteristic of M is positive, the G-action has a fixed point and it is 
therefore Poisson by a result of Prankel [Pr] ; this together with the commutativity of 
G implies that u{X, F) = for all X, F G Tp{Gp) and therefore JTp{Gp) = Np{Gp) 
for every regular point p, where Np{Gp) denotes the normal space of Gp at p. We 
denote by $ the corresponding moment map, which separates G-orbits by Theorem 
1.4. 

Let P = <I>(M) be the moment polytope of the T^-space M. Delzant constructs 
in [De] a 'canonical' smooth projective variety Mp that is G-equivariantly symplec- 
tomorphic to M. The symplectic manifold Mp is naturally equipped with a Kahler 
metric gp, a complex structure Jp and a symplectic form LUp; moreover there exists 
an antiholomorphic involution Tp on Mp such that Tpg = g~^Tp for all g E G and 
$rp = We denote by Qp the fixed point set of Tp. The compact, properly 
embedded submanifold Qp is not empty: indeed, if ^; G Mp, the fixed point set 
of G in Mp, then Tpz G Mp and ^Tpz = ^z. Since $ separates orbits, we have 
= w for all w G Mp . Hence Tpz = z. 

It follows from [Du] that ^{Qp) = <I>(Mp) since Qp is not empty. Hence Qp 
contains regular points; if q E Qp is regular, then we have the orthogonal splitting 
TgMp = Tg{Gq) © TgQp (with respect to gp) and Jp{TgQp) = Tg{Gq). 
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Abreu shows in [Ab] that the complex manifolds {M,J) and {Mp,Jp) are G- 
equivariantly biholomorphic. If (j) is such a biholomorphism, then r := (l)~^Tp(j) is an 
antiholomorphic involution of M (which is not necessarily isometric) . If Q denotes 
the fixed point set of r, namely Q := (j)~^{Qp), then for every regular point q ^ Q 
we have TgQ = JTq{Gq) = Nq{Gq). This shows that a connected component S of 
Q is an integral submanifold for the normal distribution to the regular orbits. It 
is a well known fact that E meets every G-orbit; moreover S intersects the regular 
and hence all G-orbits orthogonally. □ 

We will now give examples of torus actions, in which we can explicitly describe 
the sections. 

Example 1. We consider the complex projective space Pn(C) together with the 
polar action of the n-dimensional torus T", which is simply the maximal torus in 
the group SU(n + 1); the regular orbits are n-dimensional tori and any section is a 
totally geodesic real projective space Pn(]R)- 

Our aim is to produce new examples of polar actions on nonhomogeneous Kahler 
manifolds, which arc not hypcrpolar. In order to do this, we choose a T"-fixcd point 
p G Pn(C) and we denote by Mi the complex manifold which is obtained blowing 
up PnCC) in p; note that Mi is a projective variety and that Mi is not homogeneous 
with respect to the full group of holomorphic automorphisms (hence with respect to 
the full group of isometrics of any Kahler metric compatible with the given complex 
structure), see [Bl]. The r"-action lifts to a holomorphic action on Mi; moreover, 
since Mi is Kahler and T" is compact, we can find a r"-invariant Kahler metric g 
on Ml by averaging; we remark that the metric g has a priori no relation with the 
Fubini-Study metric on P„(C). It follows from Theorem 1.2 that the action of 
on Ml is polar. In the following we will determine the section of the action. 

We denote by vr : Mi — > P„(C) the holomorphic projection. We fix a section 
S in Pn(C) and denote by Si the closed submanifold of Mi which coincides with 
S outside {p} and has the property that Si n 7r~^{p) is Pc(rpS), where we have 
identified ir~^ip) with Pc(rpP„(C)). It is clear that Si meets every r"-orbit. We 
are now going to show that Si is a section for the T"-action with respect to the 
Kahler metric g and we will prove this in the dense open set of regular points. 

First of all we remark that the r"-action on Mi is a Poisson action, since 
the manifold Mi is simply connected (see e.g. [GS3]); moreover the T^-action 
is coisotropic by Theorem 1.4, (v). We denote by J the complex structure on Mi 
and fix a regular point y E Mi \ 7T~^{p). If we denote by Ny the normal space to 
the orbit • y, then JNy — Ty{T^^ ■ y) since the action is coisotropic. On the other 
hand, J(T^Si) also coincides with Ty{T'^ ■ y), so that Ny = Ty(T,i) and therefore 
Si is a section. We remark that the section Si, as a differentiable manifold, is the 
connected sum P„(M)#P„(M). 

Since the T^-action on Mi has fixed points in Tr~^{p), we can iterate this con- 
struction blowing up Ml in one of these fixed points. In this way, we get a sequence 
of compact Kahler manifolds M^ on which the group T" acts polarly with the sec- 
tions being the connected sum of A; -|- 1 copies of P„(M). If n > 3, then 7ri(Si) is a 
free group Z2 * Z2 * . . . * Z2 (A; -I- 1 times). Therefore it can not carry a fiat metric 
by the Bieberbach Theorem (see [Wo]). 

Remark. We point out that the lifted action of T" on Mk is polar with respect to any 
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invariant Kahler metric. We also remark that this construction yields polar actions 
on four dimensional Kahler manifolds with any nonorientable compact surface as a 
section. 

We would now like to give examples of polar actions of nonabelian Lie groups 

using the method of blow-ups. 

Example 2. We first describe the general principle which will allow us to obtain 
new examples. We consider a compact Lie group G acting polarly on a homogeneous 
Kahler manifold M with at least one fixed point p; we denote by K the isotropy 
subgroup of a regular point q (£ M. We recall the following well-known facts on 
moment maps (see [GS3] and also [HW]). Denote by $ : M ^ g* the moment 
map with respect to the given Poisson-Kahler action of G and recall that $ maps 
G-orbits onto coadjoint orbits in g*; moreover, for any regular point y G M, the 
isotropy subgroup Hy := G^^y) contains Gy and, at the level of Lie algebras, we 
may decompose f)y (with respect to a fixed biinvariant scalar product on g) as 

^y=Qy + <lj/> 

where ay = Qy {~\^y is abelian and centralizes Qy. If we denote by Ny the normal 
space to the regular orbit G ■ y and by J the complex structure on M, then 

J{Ny) = Ty{Hy.y)cTy{G-y), 

since the G-action on M is coisotropic by Theorem 1.1. It follows that the dimension 
of tty is equal to the cohomogeneity of the action of G. The tangent space to the 
orbit G-y can be identified with the complement Qy and we have the decomposition 

e = Qy + (^y+^y, 

where corresponds to the maximal complex subspace of the tangent space Ty{G- 
y). In the spirit of S. Lie and F. Engel ([LE], p. 501 ff), we call the G-action C- 
asystatic if for a regular orbit G/K there is no nontrivial complex subspace of the 
tangent space T[f,K]{G / K) which is left fixed by the isotropy representation of K. 
(For a modern treatment of asystatic actions, see [AA] and [PTl]). It follows that 
if a polar actions as above is also C-asystatic, then the space of all tangent vectors 
to a regular orbit which are fixed under the isotropy representation has dimension 
equal to the cohomogeneity of the G-action. 

We now assume that the G-action is C-asystatic and we consider the blow-up 
Ml of M in the fixed point q. If S denotes a section for the G-action on M, 
then the submanifold Si of Mi constructed exactly as in Example 1 is a closed 
submanifold which meets every G-orbit in Mi. If we now endow Mi with any G- 
invariant Kahler metric g, we show that Si is a section, namely that it meets every 
G-orbit orthogonally. Indeed, the lifted action of G on Mi is Poisson, since Mi is 
simply connected; moreover the G-action is coisotropic because Mi is a projective 
algebraic variety, G'^ acts on M, hence on Mi, with an open orbit and the same is 
true for a Borel subgroup of G"-" (see Theorem 1.4). We fix a regular point y G Mi 
and we denote by Ny the normal space to Ty{G ■ y) with respect to so that 
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JNy C Ty{G ■ y); since the action of G on M is also coisotropic, we have that 
J{TyT,i) C Ty{G ■ y) and we claim that J{Ny) = J(TyTii). Both subspaces J{Ny) 
and J{TyT.i) arc fixed under the isotropy representation of Gy and have the same 
dimension, namely the cohomogeneity of G and therefore, by the assumption that 
the action is C-asystatic, they coincide. 

As an application, we consider the quadric M := Q„ = SO(n+2)/SO(2) xSO(n). 
The isotropy subgroup G = S0(2) x SO(n) acts polarly on with sections being 
two dimensional flat tori. It is easy to check that a regular isotropy subgroup K 
is given by = Z2 x SO(n — 2) and therefore the isotropy representation splits as 
2R + 2]R'^~ , where M is a trivial submodule, while K acts on R"-2 in the standard 
way; therefore the G-action is C-asystatic. The blown up manifold Mi is acted on 
polarly by G with sections being the connected sum T^#P2(M). Note that Mi is 
not homogeneous, as already pointed out in Example 1. 

Example 3. We will now give examples of coisotropic action on complex projective 

spaces that are not polar. Let G denote the compact Lie group U(2) x Sp(n), n > 2, 
acting on C^" = C^®C^" as a tensor product of the standard representations of U(2) 
and Sp(n) respectively. This representation is not polar. (It has cohomogeneity 
three and is shown to have taut orbits in [GT], but these properties will not be 
important here.) We consider the complexification G'^ = GL(2,C) x Sp(n, C) of 
G and the corresponding representation on C^" which is multiplicity free by the 
classification of such irreducible representations of reductive noncompact complex 
Lie groups in [Ka]. This means that G'^ acts on C^"' with an open orbit as well as 
every Borel subgroup of G^. The induced action of on P4„_i(C) clearly also 
has the property that every Borel subgroup of G"' acts on P4„_i(C) with an open 
orbit. This implies by Theorem 1.4 that the action of G on P4„_i(C) is coisotropic. 
This action is not polar since otherwise the action of T^ x G on C^'*, where T^ acts 
as multiplication by scalars, would have to be polar by [PTh]; since the center of 
G already acts on C'*" as multiplication by scalars by Schur Lemma, T x G and 
G have the same orbits and therefore the G-action on P4„_i(C) is not polar. 

It is not difficult to classify coisotropic actions on complex projective spaces. If 
G is a compact subgroup of U(n + 1), then the action of G on Pn(C) is coisotropic 
if and only if the Borel subgroups of C* x G^ have dense orbits in C""*"^ which in 
turn is equivalent to the action of C* x G*^ being multiplicity free on C""*"^ (see e.g. 
[Kr], p. 199). One can now use the classification of multiplicity free representations 
of reductive groups that can be found in [Ka] for the irreducible case and in [BR] 
and [Le] for the reducible case. 



§4. Polar and coisotropic actions on quadrics 

The aim of this section is to prove Theorem 1.3, or more precisely to clas- 
sify coisotropic and polar actions on the Hermitian symmetric space Qn-2 = 
S0(n)/S0(2) X SO(n — 2) (n > 5), which can be represented as the (n — 2)- 
dimensional complex quadric in P„_i(C), i.e., the set of all totally isotropic lines in 
C" with respect to the invariant symmetric bilinear form <, > given by < z,w >= 
'^27=1 ^i'^ii equivalently as the Grassmannian of oriented two-planes in M". 
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Hyperpolar actions of compact Lie groups on irreducible compact symmetric 
spaces were classified by Kollross ([Ko]). We state his result in the special case of 
complex quadrics in the following theorem. 

Theorem 4.1 (Kollross). Let G C SO(n) be a compact connected Lie subgroup 
acting on the symmetric space Qn-2 = S0(n)/S0(2) x SO(n — 2) in a hyperpolar 
fashion. Then there exists a compact connected Lie subgroup G' of SO(ra) contain- 
ing G such that the G' -action on Qn-2 has the same orbits as the G-action and G' 

is conjugate to one of the following subgroups 

(1) G' = SO(p) X SO(g) C SO(p + q) with p, g > 1; 

(2) G' = U(n) C S0(2n); 

(3) G' = Sp(l) ■ Sp(n) C S0(4n); 

(4) G' = Spin(9) C S0(16). 

Remark. The actions listed in Theorem 4.1 all have cohomogeneity one except 
case (1) when p,q > 1 which has cohomogeneity two. Recall that a compact Lie 
subgroup of SO(n) acts transitively on Qn-2 if and only if G = SO(n) with the two 
exceptions G = G2 (n = 7) and G = Spin(7) (n = 8) as can be seen from [On], 
Theorem 2, p. 226. 

We now consider a compact connected Lie subgroup G of SO(n) acting coisotrop- 
ically on Qn-2- The complexification acts on Qn-2 with an open orbit Q and 
there exists a Borel subgroup of G'^ which has an open orbit in fi. 

In order to prove Theorem 1.3, we will make an extensive use of the results by 
Kimelfeld ([Ki]), who classified all reductive subgroups G'^ of SO(n, C) having an 
open orbit on the quadric Qn-2- 

We will distinguish between the two cases when G C SO(n) acts irreducibly on 
R" and when G acts reducibly on R". 

First case: the G-action on is irreducible 

In this case the complexified action of on C"^ can be irreducible or reducible 
(as the sum of two submodules W ® W*, where W ~ C^/^). These two subcases 
appear as separate cases in Kimelfeld's paper (see Tables I and II in [Ki], p. 535-536) 
and we too will reproduce them separately in Tables I and II below. 

As for notation, we will use the standard symbols As,Bs,Cs,Ds (s > 1) and 
(^2,-^4 to denote complex simple Lie algebras; by R{4>) we will denote the irreducible 
representation of a simple Lie algebra with highest weight and 0i will indicate 
the highest weight of the ith fundamental representation of a simple complex Lie 
algebra (the standard representation of so(n) on C" will be call simplest); by t„ we 
will indicate the Lie algebra of the ra-dimensional complex torus and e will denote 
the faithful one dimensional representation of ti . 

In the following Table I we list all complex reductive Lie algebras with the 
corresponding complex irreducible representation such that the induced action on 
the quadric Q has an open orbit moreover we will also indicate a generic stability 
subalgebra f), with p-dimensional nilradical Up, so that Q, = G'^/H, where H is an 
appropriate subgroup of G"^ having f) as Lie algebra. 
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n. 


£1 


Represen. 




1 


5d(.s),.s > :? 


Simplest 


(so(s - 1) Ci, tl) + Us-2 


2 


® Ai,s >2 


i?(<^l) X i?(</,l) 


{Cs-2 © Al e tl) + U4S-5 


3 


G2 




A2 


4 


B3 


fl(</>3) 


G2 


5 


S4 


R{<i>A) 


Bz 


6 


A2 


R(<kl+<k2) 




7 


C2 


R{2<pi) 


i2 


8 


G2 


R{4>2) 


i2 


9 


Ax 


R{Hi) 





10 


Cs 


R{4>2) 


Ai ®Ax®Ai 


11 


Fa 


RiM 


Da 


12 


eC2,s > 2 


X R{(t>i) 


Cs-2 ®Ai®Ax 


13 


Ai ®Ax 


R(4>i) X i?(30i) 






Table I 



We start noting that the cases 1,3 and 4 correspond to transitive actions of the 
compact real form G, while the cases 2 and 5 are known as cohomogeneity one 
actions (see KoUross' theorem). 

In the cases 6 to 8, the group G acts on q via adjoint representation; the coho- 
mogeneity of the G-action on the corresponding quadric Q is easily seen to be at 
least four, hence bigger than the rank of G. This means that the G-action on Q is 
not coisotropic by Theorem 1.4. 

We are now going to exclude all other cases from the above table; in order to do 
this, we use the fact that a Borel subgroup of G^ has an open orbit in G'^/H, see 
Theorem 1.4. This immediately excludes cases 9 and 13. 

The following result (see [Ki], p. 572 ff) is well known: 

Lemma 4.2. Given a semisimple complex algebraic group U and a Borel subgroup 
B, a reductive subgroup H of U has an open orbit in U/B if and only if the space 
of H -fixed vectors in every irreducible U -module is at most one- dimensional. 

Note that, if a Borel subgroup B of G'^ has an open orbit in G'^ /H, then H has 
an open orbit in G'^/B. 

In the remaining cases 10 to 12, we note that these are complexifications of 
isotropy representations of rank two symmetric spaces and that t) is exactly the 
complexification of a regular stability subalgebra for these representations. Since 
H stabilizes a two dimensional subspace of the corresponding irreducible represen- 
tation of G"', the G-action on the quadric is not coisotropic by Theorem 1.4 (v) 
and Lemma 4.2. 

We now come to the subcase that the complexified action of on C" is reducible 
(as the sum of two submodules W W*, where W C"/^). In the following 
Table II, we reproduce all such cases having an open orbit on the corresponding 
quadric, only indicating the irreducible representation W and the generic stability 
subalgebra f) for the C^-action on the corresponding quadric; we will indicate by 
Up the p-dimensional nilradical of the generic stability subalgebra f). 
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n. 





Represen. 




1 


.l,,..s > 2 




(.Is -2 C tl) + (-2^-1 


2 


As®ti,s>2 


Ri<Pl) X e 


{A,-2 t2) + U2S-I 


3 


As®Ai,s>2 


R{cf,i) X i?(0i) 


As-2 tl 


4 


© eti,s > 2 


R{4>i) X J?(0i) X e 


A5-2 t2 


5 


so(s) tl 


Simplest X e 


so{s - 2) 


6 


Ai ® tl 


R{24)i) X e 





7 


S3 ® tl 


i?(03) X e 


A2 


8 


G2 eti 


X e 


Ai 


9 


Cs,s>3 


i?(0i) 


{Cs-2 ® h) + U4S-5 


10 


Cs®ti,s>3 


i?(0l) X e 


{Cs-2 t2) + U4S-5 


11 


Aa 




Ai®Ai 


12 


^4 ® tl 


i?(</)2) X e 


Al Al tl 


13 




R{^5) 


^3 


14 


D5 tl 


R{<t>5) X e 


A3 tl 



Table II 



We start noting that case 2 is of cohomogeneity one (see Theorem 4.1) and the 
same holds for case 1: indeed, the action of U(s) on SO(2s)/SO(2) x S0(2s — 2) 
has a complex singular orbit which is Ps_i(C) = U(s)/U(l) x U(s — 1); this orbit 
also is an orbit of the semisimple part SU(s) and the stabilizer S(U(1) x U(s — 1)) 
still acts with cohomogeneity one on the normal space. 

Cases 6 to 8 and 11 to 14 can be ruled out simply because the dimension of a 
Borel subgroup of G'^ is strictly less than the dimension of the open orbit G'^/H. 

In cases 3 and 4 we consider the action of the compact group G acting on the 
quadric Q := Q4,s+2 and we will prove that in case 4 this G-action is not coisotropic; 
by Theorem 1.4 this implies that the G-action in case 3 is also not coisotropic. We 
may take G as S'(U(2) x U(s + 1)) acting on (g) C*+i ~ M4s+4. ^^len fix 
two unit vectors ei G C^, f G C*+^ and we take the oriented two plane tt in 
]^4s+4 gjygji complex line C-ei(>Siv. It is easy to sec that Gt^ is isomorphic to 

5(11(1) X U(l) X U(l) X U(s)), so that the orbit G/G^ is complex and biholomorphic 
to X Ps(C). Now, if V denotes the subspace {C ■ v)-^ C C*+^ and 62 is a unit 
vector generating (C • ei)-*- C C^, then we have 

T^Q = Hom{Tr, vr^) = vr* ®r [(d ^ V)^ + (e2 » V)^ + (C • ea ® v)^], 

which contains two pairs of equivalent 2s-dimensional G^r-submodules. Since the 
isotropy representation of G^^ on TgC^G/G.,^ contains only one 2s-dimensional sub- 
module, it then follows that the slice representation of at tt contains at least two 
distinct equivalent submodules. If the G-action on Q is coisotropic, then the slice 
representation of is also coisotropic (see [HW], p. 274), contradicting Proposi- 
tion 1.6. 

Case 5 can be ruled out because the pair (so(s) -I- ti,5o(s — 2)) is not a spherical 
pair; indeed, if it were spherical, the pair {so{s),so{s — 2)) would be spherical too, 
contradicting Lemma 4.2. 

In case 9, we show that the action of G = Sp(s) is not coisotropic. We consider 
the G-action on W ~ C^* ~ M^* and the corresponding action on the quadric 
Q '■= Qis-2- We fix an oriented two plane tt in M^*, which is invariant under 
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the complex structure of C^*. Then = x Sp(s — 1) and the orbit G ■ tt is 

difFeomorphic to a complex projective space P2s-i(C). The tangent space T-,^Q can 
be identified with Hom{'K, tt^) and we have the orthogonal splitting tt"*" = tt + W , 
where tt + tt is the quaternionic span of tt, while W ~ M^(®~^) is the standard 
representation space of Sp(s — 1). 

If we denote by p the representation of the center of G^^ on vr, then acts on 
TT by means of p~^; the tangent space T^^Q splits as a G^r module as 2M + ]R^ + 2W , 
where M denotes the trivial module, is acted on by by means of p^, while W is 
the standard representation space of Gtt- Since the isotropy representation of G-^ on 
the orbit G • vr is + W ^ we see that the normal space N^^ is isomorphic to 2M + W 
as a G^r-module; it follows that a regular isotropy subgroup K of the G-action on 
Q is isomorphic to x Sp(s — 2) of corank one in G, while the G-cohomogeneity 
on Q is three. This proves that the G-action on Q is not coisotropic. 

In case 10 the group G = x Sp(.s) acts on C^''' — M'*''' in a standard way; 
for notational reasons we will denote by Z the one dimensional center of G. We 
will show that the G-action on Q := Q4S-2 is coisotropic but not polar. As in 
the previous case, we fix an oriented two plane tt C M'** which is Z- invariant; 
the isotropy subgroup G^r is Z x x Sp(s — 1) and the orbit O := G • tt is a 
complex projective space P2s-i(C). If we compute the slice representation along 
the same lines as in case 9, we see that the normal space splits as a G^r-module as 

+M'*(*~^) , where Gjr acts nontrivially on each factor (this action can be shown to 
be actually polar). A regular isotropy subgroup K is isomorphic to x Sp(s — 2) 
of corank two in G and the cohomogeneity is two, hence the G-action on Q is 
coisotropic. 

We now show that this action is not polar. It is not difficult to see that there 
exists a totally geodesic submanifold F C Q with T^F = A^^r such that F is isometric 
to the complex projective space P2s-i('C) after an appropriate rescaling of the 
metrics. Moreover we observe that the group G lies in U(2s), which acts on Q with 
cohomogeneity one; the two singular U(2s)-orbits are O and O* := U(2s) • tt* = 
G • TT*, where tt* denotes the two-plane tt with reversed orientation. The orbit O* 
consists actually of all points of maximal distance d from O and d is the diameter of 
F. Now suppose that the G-action is polar. If S is a section passing through vr, then 
T, C F and we know from [PTh] that S is a real projective plane. The submanifold 
S contains a circle S of points at distance d from tt, hence S is contained in O*; 
this means that S intersects the G-orbit in a circle, hence not orthogonally; a 
contradiction. 

So far we have proved the following 

Proposition 4.3. Let G be a compact connected Lie subgroup of SO{n), where 
n > 5. Suppose the action of G on R" is irreducible. Then the action of G on 

Q := Qn-2 is nontransitive and polar if and only if it is hyperpolar and G = 
\J{n/2),SU{n/2),G = Sp(l) • Sp(n/4) or G = Spin(9) C S0(16). 

The action of G on Q is nontransitive and coisotropic if and only if the action 
is polar or G = T^ ■ Sp(n/4) 

Second case: the G-action on M" is reducible 
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We will now suppose that the linear G-action on M" is reducible. Here the proof 
will be more direct than in the first case. We will first deal with the particular case 
when G has a fixed nonzero vector e in M", so that we can write = M • e + R"""^ 
(note that we are not supposing that R""-*^ is G-irreducible) . Wc prove the following 

Proposition 4.4. Let G be a compact connected Lie group acting on V := R"^ with 
nonzero fixed point set. Then 

(i) the action of G on Q := Qn-2 is coisotropic if and only if the fixed point 

set is one dimensional and G acts transitively on the unit sphere of 
(y'^)^ ~ R""^ with the exception of G = Sp(fc) acting in a standard way 
on R^'=,n = 4A; + 1. 

(ii) the action of G on Q := Qn-2 is polar if and only if the fixed point set V'^ 
is one dimensional and G = SO(n — = G2 (n = 8) or G = Spin(7) 
(n = 9); in particular the action of G is polar if and only if it is hyperpolar. 

Proof We suppose that the G-action on Qn-2 is coisotropic. We first show that G 
must act transitively on the unit sphere of R"^~^. We fix a nonzero vector v € R""-*- 
which is regular for the G-action on R"~^ and consider the oriented two plane 
TT = e A v; we note that G^ = G-,^. Wc may identify the tangent space T^^Q with 
e AW + V A W, where W C R"~^ is the set of all vectors orthogonal to v in R"~^; 
moreover T-^G ■ tt is given by 

T^G ■ TT = {Xe Av + eA Xv; X e q} = {e A Xv; X G g}. 

We note that q-vcW and we split orthogonally W = q ■ v + V, where dim V + 1 
is equal to the cohomogeneity of G in R"~^. Then 

T^Q = T^G -TT + eAV + v AV + v Aq-v. 

We recall then that the complex structure J on T^Q is given (up to sign) by 

J(e A wi + w A ^2) = e A W2 — f A w;i , 

for wi,W2 G W. Therefore the subspace eAy + i;Ayisa complex subspace of 
(r^G • tt)-*- on which G,r acts trivially. The slice representation of G,r is coisotropic 
since the action of G on Qn-2 is coisotropic (see [HW], p. 274) and can therefore 
not leave a nontrivial subspace invariant. This implies that that eAV+vAV = {0}, 
hence y = {0} and our first claim follows. 

We now use the fact that G induces a coisotropic, possibly transitive action on 
the subquadric Q(M."'~^) = Qn-3- An inspection of the lists of compact groups 
acting transitively on spheres (see e.g. [Bo]) and quadrics (see the remark after 
Theorem 4.1) together with Proposition 4.3 shows that G can be either 

(a) G = SO(n — 1), G2, or Spin(7) which all give rise to a cohomogeneity one 
action on Q (those of G2 and Spin(7) being orbit equivalent to the one of SO(n — 1) 
for an appropriate n), 

(6) G = SU(i^), U(^), Sp(l) • Sp((^), T^ . Sp((^) or Spin(9). 

It is easy to check that all groups listed in (b) yield coisotropic actions; their 
cohomogeneity on the quadric is given by the cohomogeneity of Gy acting on q- v. 
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We will now show that the G-action on the quadric is not polar in all the cases 
enumerated in (6). Let now G be one of the groups listed in (6); we already know 
that the induced action of • Sp( ^""4 ) on the subquadric Qn-3 is not polar if 
n > 5, while, for n = 5, G coincides with U(2), so that this case can be ruled 
out. Using the same notation as above, it is easy to see that G^ acts on q ■ v 
polarly and with cohomogcncity two. We choose two orthogonal vectors wi,W2 of 
unit length in q ■ v which span a G^j-section in q ■ v; if the G-action on Q is polar, 
then there exists a section E passing through tt with T^S = Span{v Awi,vAw2)- 
The unique totally geodesic submanifold of Q with such properties is given by 
T, = {v A (ae + bwi + CW2); a? + 1)^ + = 1}. If we now consider the two plane 
TTi = V Aw\, then r^r^S is generated by v A e,v A W2, while T^^G • tti is given by 
Qv A Wi + V A QWi. Since the scalar product < v A QWi,v A W2 >=< QWi,W2 > is 
not zero, we see that E cannot be a section and this contradiction proves our last 
claim. □ 

We now suppose that G acts on reducibly without fixed vectors and that 
it acts on the quadric Qn-2 coisotropically. We split R" = Vi © V2 into two 
nontrivial invariant subspaccs with dim Vi >2 and consider first the case when both 
Vi are G-irreducible. The restriction of the g-action onto Vi defines homomorphisms 
Pi : g ^ so{Vi) for z = 1,2 . We put Qi := pi(g), i = 1,2, so that g C gi + 52- 
If dimFj > 5, Proposition 4.3 and the fact that the action of G restricted to Vi 
induces a coisotropic action on the corresponding quadric imply that 0j is one of 
the following Lie algebras 

so(p),02,spin(7),spin(9),5u(p),u(p),R + sp(p),sp(l) +sp(p) (*) 

for some integer p. On the other hand, if 2 < dimV^ < 4, the list in (*) contains 
all possible subalgebras of so{Vi) which act on Vi irreducibly, so that (*) gives the 
list of all possible candidates for Qi, i = 1,2. 

Lemma 4.5. We have Qi ^ su{p),u{p),'R + sp{p) . 

Proof. We suppose that gi € {5u(p), u(p), M + 5p{p)}. We fix an oriented two 
plane vr in V2 and let denote the stability subalgebra. If we identify T^Q with 

Hom{7T,7r^), we see that the slice representation at vr contains a submodule iso- 
morphic to Hom{7T, Vi). Let Jo and Ji denote g7r-invariant complex structures in vr 
and Vi respectively and denote by Wk, k = 1,2, the gTr-submodules of Hom{7T, Vi) 
consisting of elements / such that f{Jov) = (— 1)*^ Ji/(u) for all v S Vi. It is easy 
to see that Wi and W2 are equivalent g^r-modules. This implies that the G-action 
cannot be coisotropic by Proposition 1.6. □ 

We now prove the following 
Lemma 4.6. 7/g is not isomorphic to gi, then Q2 C. g. 

Proof. Since g is not isomorphic to gi, the kernel of pi, given by g PI g2, is not 
trivial. Therefore we can write g = gi + (g n g2), sum of two ideals of g, where 
gi is isomorphic to gi. Now p2(g n g2) is an ideal of g2; if g2 is simple, then 
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P2(0n02) = 0n02 = 02, hence 02 C q. If 02 is not simple, namely 02 ^ sp{l)+sp{p) 

and 02 Q, then ^2(01) is a nontrivial ideal of 52- We note that P2{Qi) cannot 
coincide with g2- indeed, if so, we have gi ~ sp(l) + sp{p) ~ 32 and P2|gi is an 
isomorphism; since kei p2 = n gi is an ideal of g and 0ngingi = {O}, we have 
kerp2 = {0} and dimg = dimg2 = dimgi, forcing g ~ gi, a contradiction. 
Therefore we have the following possibilities: 

(a) P2(gi) = sp(l). Then g H g2 = sp{p). Now gi ~ gi contains an ideal 
isomorphic to sp(l), hence we have the following subcases: 

(al) g = sp(l) +sp{p), where pi(g) = 5p(l) acting on by Lemma 4.5, P2(g) = 
spil) + spip); 

(a2) g = sp(l) +sp(p) +5p{q), where gi = sp(l) + sp{q) and g2 =5p(l) +sp{p). 

(b) P2(0i) = sp(p). Then we have the following subcases: 

(bl) g = sp(l) +sp(p), where gi = sp{p), p = l,2; 
(b2) e = sp(l) + sp{p) +sp(l) and gi ~ 02. 

We now show that the above cases do not yield coisotropic actions. Indeed, for 
(al), if we fix a two plane vr C M"^ = Vi, then P2(07r) C M + sp(p) leaves a complex 
structure invariant and the same arguments as in Lemma 4.5 can be applied. Cases 
(a2) and (b2) can be dealt with similarly. As for (bl) a simple computation shows 
that the cohomogeneity c of G is at least 4 if p = 1 and 9 if p = 2, contradicting 
the fact that c does not exceed the rank of G. □ 

Therefore, if is not isomorphic to 0j for i = 1,2, then = 0i + 02- 

Lemma 4.7. // = 0i + 02, then 

(i) the action of G on Qn-2 is coisotropic if and only if 

Qi G {so{Vi),<52,spm{7),spin{9),sp{l)+sp{p)} 
fori = 1,2; 

(ii) the action of G on Qn-2 is polar if and only if 

Qi G {so(Vi),(52,spin(7)} 

fori = 1,2. 

Proof, (i) We use the list (*) together with Lemma 4.5. 

Now if gi G {5o{p), (S2,spiTi(7)} for i = 1, 2, then it can be easily checked that G 
has the same orbits in the quadric Q = Qn-2 as SO(Fi) x 80(^2), which acts on 
Q hyperpolarly, hence coisotropically. 

Now we suppose that, say, gi = spin(9). We choose an oriented two plane tt C V2 
such that the orbit G-tt is complex; then the stability subalgebra g^ = spin(9)+(g2)7r 
and rank(g2)7r = rank(g2) since G • tt is a flag manifold. 

We first claim that the g-^-action on tt is not trivial. 

Indeed, when g2 is so(V2),(S2 or spin(7), then our claim follows from the fact 
the groups SO(V2),G2 and Spin(7) act transitively on the Stiefel manifold of two 
frames in V2 (see e.g. [On], p. 90, 93). When g2 = spm(9), then the action of Spin(9) 
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on Qi4 has cohomogeneity one and we claim that there exists only one complex 

singular orbit in Q14 given by G-vr = Spin(9)/U(4), where U(4) acts nontrivially on 
TT. Indeed, we see the spin representation of Spin(9) as the isotropy representation 
of the Cayley plane P2(0) = F4/Spin(9) at the origin o and, if we choose a two 
plane tt inside the tangent space = at o of a projective line in P2 (O) through 
o, then the stabilizer Spin(9)7r leaves W invariant, since two different projective 
lines cannot be tangent along a two plane; this means that Spin(9)7r is contained 
in Spin(8) and therefore it covers x S0(6) C SO{W). Hence Spin(9)^ = U(4) 
and it acts nontrivially on it. Moreover, since xiQu) = 16 is the sum of the Euler 
characteristics of the singular orbits in Q14, and since x(Spin(9)/U(4)) = 16, we 
see that there is exactly one complex singular orbit G ■ tt = Spin(9)/U(4). When 
02 = sp(l) + sp{p), then the action of Sp(l) • Sp(p) on Q4p_2 has cohomogeneity 
one; if vr is a complex line, viewed as a real two plane, inside a quaternionic line 
in MP, then it is easy to sec that (02)77 = + 5p{p — 1) acts nontrivially on vr. 
The same argument as in the previous case shows that Sp(l) • Sp{p) has only one 
complex singular orbit in Q4p-2- 

So we may write (02)77 = K + ()2, where ()2 denotes the kernel of the (02)7r-action 
on vr. The slice representation of (02)77 on splits as (M?)* 0Vi +N' , where N' is 
acted on by M + f)2; note that N' = {0} if and only if 02 G {so(p), (62,spin(7)}. Now 
(M^)* (g)Fi is acted on by M+spin(9), while f)2 acts trivially on it; the representation 
of M+spin(9) on (M^)*(8)Vi is coisotropic, sec [Ka] or notice that a regular subalgebra 
is given by su(3) of corank three and the representation has cohomogeneity three 
(see [Ya], p. 324). If 02 G {so{p),&2,spin(7)}, then the G-action on the quadric Q 
is coisotropic; indeed, a regular isotropy subalgebra of is su(3) + [)2 of corank three 
in 07r, hence of corank three in 0, while N' = {0} implies that the cohomogeneity 
of the G-action on Q is the cohomogeneity of M + spin(9) acting on (M^)* ^ Vi. 

If 02 = spitt(9), then (02)77 = 1^ + su(4), N' = and su(4) acts on it by 
cohomogeneity one; hence a regular isotropy subalgebra of is su(3) + 5u(3) of 
corank four, while the cohomogeneity of the G-action is also four. 

If 02 = sp(l) +sp(p) {p > 2), then (02)^ = +sp(p - 1) and A^' = M^f"! 
So (07r) = 5u(3) + M + sp{p — 2) of corank four in and the cohomogeneity of the 
G-action on Q is also four. 

Next we suppose that 0i = sp(l) +sp{p). If we argue in the same way as in 
the case 02 = spin(9), we see that we only need to prove that the representation of 
M + 0p(l) +5p(^>) on (M^)* (g)M''P is coisotropic (see also [Ka]). This representation 
has cohomogeneity three (see [Ya], p. 318); a regular isotropy subalgebra t contains 
a copy of 5p{p — 2) as an ideal, so that we can write 6 = + sp(]? — 2) and a simple 
dimension count shows that t' = R. So the corank of 6 in R + sp(l) + sp(j3) is also 
three and the representation is coisotropic. 

(ii) Using the same arguments as in case (i), we only need to show that the 
representation of M + spin(9) on ^ ]^i6 ^^^^ of R + sp(l) + sp{p) on R^ (g) R^p 
are not polar. Actually these representations are neither isotropy representations 
of irreducible symmetric spaces nor appear in the list of irreducible polar represen- 
tations which are not isotropy representations of symmetric spaces (see the table 
from [EH] in the proof of Lemma 2.7). □ 

Next we suppose that is isomorphic to 0i. 
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If 01 is simple, then pi,p2 are isomorphisms and Qi ~ 02- In particular, Qi G 
{so(Vi), (S2,spin(7),spin(9)}. Now, if pi is equivalent to p2, we select a two plane 
vr C Fi and observe that the slice representation of contains ifom(7r, 1^) as 
a complex submodule; if we denote by tt the same oriented two plane in V2 ^ 
Vi, then Hom{'!T,Tr) contains a trivial gTr-submodule. This means that the slice 
representation contains a trivial complex submodule, preventing the G-action from 
being coisotropic. If on the other hand pi is not equivalent to p2, then g = spin(7) 
or g = 5pin(9); in this case a simple computation shows that the dimension of a 
Borel subalgebra of g"^ is strictly less than the complex dimension of the quadric 
and the action is therefore not coisotropic. 

If gi is not simple, i.e. gi = sp(l) + sp(p), then g2 can be sp(l) ~ so(3), 
sp(2) c:i SO (5) or sp(l) +sp{p). The first two cases already appeared in the proof of 
Lemma 4.6 and the last case can be ruled out with the same arguments. 

In order to complete the proof of Theorem 1.3, we need to prove the following 

lemma. 

Lemma 4.8. Let G be a compact Lie group acting reducibly on (n > 5) and 
coisotropically on the quadric Qn-2- Then the number of irreducible submodules of 
M" is at most two. 

Proof. We assume that contains at least three irreducible submodules Vi , V2 , V3 , 
none of which is trivial. We then restrict ourselves to the coisotropic action of G on 

the quadric corresponding to the G- module V = Yl'i=i *^^n apply the results 

obtained so far to the quadrics corresponding to the sums Vi + Vj, i,j € {1,2,3}, 
showing that g = gi + g2 + Qs, where each Qi G {so{Vi), 62,spin(7),spin(9),sp(l) + 
sp{p)}. We consider a two plane vr C Vi and its isotropy subalgebra = (gi)7r + 
02 + 93- The slice representation of g^^ contains Hom{TT,V2 + V3) as a complex 
submodule which is coisotropic; this means that g^ acts nontrivially on vr, since 
otherwise Hom{Tr, V2 + V3) would contain equivalent 0,r-submodules, contradicting 
Proposition 1.6. The action of g^r on Hom{Tr, V2+V3) is the action of t) := M+g2+g3 
on {M.'^)*^{V2 + V3). Wc set Wi := {R'^)*0Vi and t,; the regular isotropy subalgcbras 
of M + gj acting on Wi for i = 2, 3; it is known that the regular isotropy subalgebra 
of M + so(n) acting on (8> R" is so(n - 2), hence 6j C Qi for i = 2,3. Now a 
regular isotropy subalgebra ^ of f) acting on Hom{Tr,V2 + V3) is given by 61 + ^2- 
Since the R + gj-actions on Wi are coisotropic, we have that the cohomogeneity c 
of the f)-action on Hom{Tr, V2 + V3) is given by 

c = dim(I'Fi + W2) - dim(gi + g2) - 1 + dim(ti + 62) 

= dim Wi - dim(R + gi) + dimti 

+ dim W2 - dim(R + g2) + dim 62 + 1 

= rk(R + gi) - rk{h) + rk(R + g2) - rk{h) + 1 

= rk(R + gi + 02) - rk(li + 62) + 2. 

This means that the Gt^ action on Hom{Tr, V2 + V3) is not coisotropic. Hence the 
G-action on Qn-2 is not coisotropic. □ 
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